1. Field of the Invention
The present invention relates to an apparatus and a method of processing an image data, a transmission medium and a recording medium. In particular, the present invention relates to an apparatus and a method of processing an image data, a transmission medium and a recording medium, capable of reading an accompany information much accurately.
2. Description of the Related Art
There is a technology, for a specific image data (a still image or a motion image sequence), which adds an information accompanied therewith into an image data, detects and utilizes the accompany information at a time when reproducing. As a representative example thereof, an addition of a copyright information is given.
When an unspecified (a general) user can utilize a specific image data, it is necessary to add a copyright information into the image data in advance in order that a person having a copyright for the image asserts the right thereof. By adding the copyright information, it makes possible to take steps of not displaying the image data when the copyright information indicating the image data to be not displayed is detected in the reproducing apparatus of the image or during the procedures in the reproducing method.
The above-mentioned addition or detection of the copyright information is, at a present time, well utilized for preventing an unauthorized duplication of videotape and the like. Recently, there exist many video tape rental shops, but if many users had enjoyed the video tapes by making the unauthorized duplications of the video tapes that they have rented by a low fare from the rental shops, the damages of the persons having the copyright of the video tapes and the video rental shops would be very bad.
Since an image data is recorded analogously in videotape, an image quality will be a little degraded when duplicating. As a result, when the duplications have been made several times, it becomes very difficult to maintain the image quality that has been held originally.
In contrast, in the devices that record and reproduce the image data digitally, the damage caused by the unauthorized duplication could be much worse. That is, in the devices that dealt with the image data digitally, there is no degradation of the image quality by duplication, in principle. As a result, the prevention of the unauthorized duplication in the devices that process digitally is far important than the one in analogue.
There are two methods of adding the information accompanied with the image data into that image data.
A first method is a method of adding into an auxiliary part of the image data. For example, in the videotape, as shown in FIG. 18, an auxiliary information of that image data is recorded in an upper part of the screen (the auxiliary part) which is substantially not displayed on the display screen. It is possible to add an accompany information, by utilizing a part of this area.
A second method is a method of adding into a-main part (a part to be substantially displayed) of the image data. This is, as shown in FIG. 19, the one that adds a specific pattern (WaterMark) to a part of or an entire image, with a degree of not visually sensible. As a concrete example thereof, there are a spectrum diffusion and the like that implement an addition or a detection of the information, by utilizing a key pattern which is generated using random numbers or M-series and the like.
In the following, an example of an addition or a detection an accompany information to the main part of the image data when using a watermark pattern. The concrete examples are shown in FIGS. 20-23.
Assuming to use the watermark pattern corresponding to the pixels of 4nxc3x974n, as shown in FIG. 20. The watermark pattern takes either a plus or a minus of 2 symbols for each pixel.
The watermark pattern preferably takes either one of 2 symbols at random, and a shape and a size of that area are optional.
When adding the watermark, an area whose size is equal to the size of the area of the watermark is set on the image subjected to implement the addition. By checking the area being set against the watermark pattern in superimposition, the value a is added to a pixel to which a plus symbol corresponds, and the value b is subtracted from a pixel to which a minus symbol corresponds. The values a, b might be arbitrary values, but they should be kept as constants throughout the watermark pattern thereof.
In the examples of FIGS. 21-23, a=1, b=1 are set, and as shown in FIG. 21, when all pixel values in the area subjected to implement the addition are 100, the pixel values of 101 and 99 are formed by an embedding operation which corresponds to the watermark.
When detecting the watermark, an area whose size is equal to the size of the area of the watermark is set on the image subjected to implement detection. A value of which all of the pixel values of this area are summed up is used as an evaluation value. When summing all of the pixel values, an addition is applied for a pixel to which a plus symbol corresponds, and a subtraction is applied for a pixel to which a minus symbol corresponds, by checking the area being set against the watermark pattern in superimposition. In the example of FIG. 22, a pixel of the pixel value 101 is added, and a pixel of the pixel value 99 is subtracted, and the calculated results thereof are added. At this moment, the watermark can not be detected correctly unless using the pattern that is the same as the watermark pattern used for adding the watermark. According to the detection operation described above, for example as shown in FIG. 22, the evaluation value at a time when the watermark is added turns to be (4n){circumflex over ( )}2 (the same as the numbers of pixels included in the area), and as shown in FIG. 23, the evaluation value at a time when no watermark is added turns to zero (0).
When the area of the watermark pattern is a quite large, and the watermark pattern is sufficiently at random, the evaluation value at a time when no watermark is added turns to be almost zero all the time. As a result, when the evaluation value exceeds a certain threshold, it might be evaluated as the watermark being added. According to the procedures described above, it makes possible to add a binary information (1 bit) of whether or not the watermark is added. When it is desirable to add more information, the information of the 2{circumflex over ( )}k ways (k bits) can be added by dividing the whole image into the k-areas and according to a method of processing such as respectively implementing the above mentioned operations.
As a watermark pattern, the one that is generated by using, for example, the M-series can be utilized. The M-series (the longest code series) is a sequence of numbers consisting of the binary symbols of 0 and 1, and the statistical distributions of the 0 and 1 are constant, and the code correlation is 1 at the origin, and is inversely proportional to the code length at other points. Of course, a watermark might be generated by a method other than the M-series.
When recording and reproducing. the image data digitally, it is a common practice to compress the data because an amount of information thereof becomes quite large if being kept as it is. As a method of compressing the image data, the low bit rate coding methods, such as the JPEG (Joint Photographic Experts Group) (color still image coding method), or the MPEG (Moving Picture Experts Group) (color motion image coding method) and the like are internationally standardized and turned to be in practical use. In the following, as an example, considering to compress the image data by the low bit rate coding, a structural example of an addition and a detection of an accompany information will be described.
In FIG. 24, a configuration of an encoder is shown. In the image data inputted, at first, in the watermark adding device 1, an accompany information signal f is referred. Referring to this accompany information signal f, it is decided whether or not a process of adding a watermark is implemented, and thereafter, the accompany information signal f image data is inputted into the encoder 2, and the sign bit train is generated by implementing the low bit rate coding.
In FIG. 25, a configuration of a decoder is shown. The sign bit train inputted is decoded to the image data in the decoder 21. Thereafter, the accompany information signal f is detected in the watermark detector 22.
When the image data is not low bit rate coded, it needs to use a configuration having neither the encoder 2 of FIG. 24 nor the decoder 21 of FIG. 25.
The configurations of the watermark adding device 1 and the watermark detector 22 are shown in FIGS. 26 and 27, respectively.
In the watermark adding device 1 of FIG. 26, the image data inputted is added with a watermark in the watermark image generator 11, using a watermark pattern being recorded in the watermark pattern holding memory 12, when the accompany information signal f is turned on. The watermark image generator 11 outputs the image data inputted just as it is, when the accompany information signal f is turned off. In this case, the accompany information signal f is, for example, a signal of 1 bit and the like, and it is only needed to evaluate an information of on/off thereof.
A sequence of processes to be implemented in the watermark adding device 1 is shown in FIG. 28. At first, in step S81, setting predetermined values to the addition levels a, b of the watermark. The values of a, b could be arbitrary values, but the same values are used until the processes for all pixels of the subjected areas are completed. In step S82, setting the area whose size is equal to the size of area of the watermark pattern on the image subjected to implement an addition of the watermark, and checking against the watermark patterns for each of the pixels within the area. In step S83, implementing an evaluation of a symbol of the watermark, and when the symbol of the watermark to which the pixel corresponds is a plus, then in step S84, a is added to the pixel. When the symbol of the watermark to which the pixel corresponds is a minus, then in step S85, b is subtracted from the pixel. This process is, in step S86, repeated until it is determined to be implemented for all pixels in the subjected area.
In the watermark detector 22 of FIG. 27, for the image data inputted thereto, the evaluation value is calculated in the evaluation value calculation device 31, using the watermark pattern recorded in the watermark pattern holding memory 32. The watermark pattern stored in this watermark pattern holding memory 32 is the same watermark pattern stored in the watermark pattern holding memory 12 of the watermark pattern adding device 1. The calculated evaluation value is threshold-processed in the evaluation value comparator 33, and an accompany information signal f is outputted therefrom. Further, the image data inputted is outputted as just it is, or as being implemented with a processing or a predetermined process. This will be described later.
FIG. 29 shows a series of processes to be implemented in the watermark detector 22. At the beginning, in the step S91, settings of the initialization of the evaluation value sum and the threshold th are implemented. In the step S92, setting an area equal to an area of the watermark pattern in size, and verifying each pixel within the area with the watermark pattern. When the symbol of the watermark to which the pixel corresponds is determined as a plus, in the step S93, then, in the step S94, the pixel value thereof is added to the evaluation value sum. When the symbol of the watermark is determined as a minus, then, in the step S95, the pixel value thereof is subtracted from the evaluation value sum. These processes are repeated until it is determined that the processes are implemented for all pixels in the subjected area. Thereafter, in the step S97, comparing the evaluation value sum with the threshold th, and if sum greater than th, it is assumed that the watermark is added, and then in the step S98, turning on the accompany information signal f. If not, then, in the step S99, turning off the accompany information signal f.
The accompany information signal f is, when preventing an unauthorized duplication, is utilized as following. For example in case of the decoder of FIG. 25, the image data and the accompany information signal f outputted are passed to an image display unit (not shown). In the image display unit, when the accompany information signal f is on, it. is displayed as just it is, but when the accompany information signal f is off, the processing or the processes such as, for example, not displaying the image, not displaying the main area of the image data, scrambling the image (i.e., displaying the received image data randomly), are implemented. Or, providing the image converter 34 shown in the watermark detector 22 of FIG. 27, it may be configured to implement such processing or processes of the image data according to the accompany information signal f.
In the above described two different methods of adding the information accompanying with the image data to the image data thereof, there are the problems as described below.
In the first method of adding an accompany information to an auxiliary part of the image data, when the auxiliary part to which the accompany information is added is ignored, it becomes difficult to prevent the problems such as the unauthorized duplication and the like. For example, when reading the image data recorded digitally into a commercial personal computer, and copying the main part by cutting out the main part only with ignoring the auxiliary part, an image quality thereof becomes completely the same as the one before copying. In this case, the meaning of adding the accompany information to the auxiliary part is completely lost.
In the second method of adding an accompany information to a main part of the image data, there is no such a case that the added accompany information is to be diminished, by the copying procedures and the like as described in the above-mentioned first method, for example. However, when implementing the various kinds of signal processes such as the noise reduction filters and the like for the image data, the added accompany information component is damped and thus it becomes inextractable.
In particular, when the original image data themselves are compressed using the low bit rate coding such as the JPEG or the MPEG, there is so much harmful influence to be exerted as a result of the quantization process thereof. The added accompany information component is amplified to a degree of visually detectable, by the quantization process of the low bit rate coding, and thus the image quantity thereof is degraded, or damped to a degree of inextractable, thereby the original meaning thereof is to be lost.
In order that the components of the accompany information are not changed by these signal processes, there is a method of adding the accompany information using a special area in the image. However, since such area exists only in a portion of the entire image sequence, the area of the watermark pattern can not be taken as sufficiently large. As a result, even when no accompany information is added, the evaluation value turns to be large other than 0, so if using an absolute evaluation reference that evaluates as the accompany information is added when it exceeds a certain threshold, the detection of the accompany information becomes very difficult.
Further, when the area to which the addition of the accompany information is implemented is merely a portion of the entire image sequence, it becomes extremely difficult to add a plurality of information. For example, when adding the accompany information by dividing the entire image into the k areas, an area of the watermark pattern for each area becomes narrower according to the number of areas, thereby the accompany information becomes almost undetectable.
The above-described problems are more pronounced in the motion image sequence, in particular.
It is an object of the present invention to enabling to indisputably detect an accompany information without exerting much influence on the original image data, in a view of such situations described as above.
The present invention relates to an apparatus and a method of processing an image data, a transmission medium, as well as a recording medium. In detail, in an image data processing of processing an image data in which an accompany information is embedded as a watermark, having a memory which stores the patterns of the watermark, setting a phase of a pattern of the watermark being stored in the memory, to a first phase or a second phase, calculating a first evaluation value corresponding to a pattern of the watermark of the first phase, of the image data inputted, and a second evaluation value corresponding to a pattern of the watermark of the second phase, and comparing the first evaluation value, and a value corresponding to the second evaluation value.
Further, when an accompany information is to be embedded as a watermark for an image data, it has a first memory for storing a pattern of the watermark; a second memory for storing a relationship of a plurality of accompany information signals and a phase of said watermark; and then selecting a phase of the watermark corresponding to the accompany information signal inputted; reading a pattern of the watermark being stored in the second memory, and for setting to a phase selected by the selecting means; and adding a watermark of the phase set to the image data.
Moreover, in case of processing an image data into which an accompany information is embedded as a watermark, having a first memory for storing a pattern of the watermark, having a second memory for storing a relationship of a plurality of accompany information signals and a phase of the watermark, setting a phase of a pattern of the watermark being stored in the first memory, to a first phase and a second phase, comparing the first evaluation value corresponding to a pattern of the watermark of the first phase, of the image data inputted, and the second evaluation value corresponding to a pattern of the watermark of the second phase, comparing the values corresponding to the first and the second evaluation values, and then, selecting an accompany information signal from the second memory in accordance with a result of the comparison.